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Abstract 

In this paper we present a new Turbo analog error correcting coding scheme for real valued 



o 
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q ■ signals that are corrupted by impulsive noise. This Turbo code improves Donoho's deterministic 



construction by using a probabilistic approach. More specifically, our construction corrects more 
errors than the matrices of Donoho by allowing a vanishingly small probability of error (with the 
increase in block size). The problem of decoding the long block code is decoupled into two sets of 
parallel Linear Programming problems. This leads to a significant reduction in decoding complexity 
as compared to one-step Linear Programming decoding. 
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■^j- ■ I. Introduction 

In this paper we discuss the problem of error correcting codes with real valued entries. The goal 
is to recover an input vector m G M. k from a corrupted measurement vector y = Gm + e, where 
G 6 M. nxk is a (coding) matrix that has full column rank (n > k , R := - is the code rate) and 
e G M n is a (sparse) error vector. If the vector e is known, then y = y — e = Gm and since G has 
full column rank, m can be easily reconstructed from y. Thus, reconstructing m from y is equivalent 
to reconstructing e from y. By constructing a parity check matrix H [T| which eliminates G (i.e. 
HG = 0) we obtain the syndrome s which is defined as 

s = Hy = HGm + He = He (1) 

Note that the syndrome s depends only on the error vector e and not on the input vector m. Let 
r = n — k be the redundancy of the code. Since G £ M nxk is a full column rank matrix, its kernel 
has dimension r, thus H S M rxn . 
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The sparsity requirement of e is intuitively explained by the fact that if the fraction of the corrupted 
entries is too large the reconstruction of m is impossible. Therefore, it is commonly assumed that 
only a few entries of e are non-zero 

\\e\\ io :=\{i: ei ^0}\<t{n). (2) 

Given the coding matrix G, it has been shown in that if t(n) > cos P a 2 rk ( G ) it i s impossible to 
recover m from y, where the cospark of matrix A is defined as 

cospark(A) := min IIAxL . (3) 

This provides an upper bound on the number of errors that can be corrected. In a way, the cospark is 
the analog equivalent of the Hamming distance between the codewords. In |2 ] it has been shown that 
cospark(G) = spark(H), where the spark of a matrix is the minimal number of linearly dependent 
columns of a matrix: 

spark(H) := min llxIL subject to Hx = 0. (4) 

From (01) it is easy to see that the largest number of correctable errors cannot exceed the rank of the 
parity check matrix. We assume that the error vector e is the sparsest vector that explains the input 
y. Therefore, the decoding problem is reduced to finding a sparse solution to the underdetermined 
system: 

min llxIL subject to Hx = s. (5) 

This problem is NP-hard O. 

The performance of the code depends on the coding matrix G (or alternatively the parity check 
matrix H) and the decoding technique. Wolf [4] extracts r = 2t columns from the IDFT matrix and 
uses it as a coding matrix G. Therefore, after the encoding we get a sequence of real (or complex) 
numbers y = Gm, whose DFT has zeros in certain positions. He showed that the same technique for 
decoding BCH codes over finite field can be utilized to decode the real number code as well. He also 
showed that these decoding algorithms are tolerant to small errors at every entry in addition to the 
impulsive noise. Further work on these real BCH codes has been done by Henkel [5]. He studied the 
influence of small additive noise on the detection of error locations by algebraic methods. In addition, 
he provided another proof of the main result of [4] based on the Newton interpolation method. In 
this proof a different definition of syndrome is presented that make it possible to locate an error-free 
range of the codeword by observing this new syndrome (i.e. without any further operations). 

There are many applications for analog coding. Gabay et al. showed that a real BCH code 
can be used for simultaneous source coding and impulse noise cancellation. More specifically, they 
showed that simultaneously correcting the impulse noise and reducing the quantization noise by using 
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BCH codes leads to a reduction in the end-to-end Peak Signal to Noise Ratio (PSNR). Henkel 
showed that using Wolf analog codes (also known as Analog Reed Solomon Codes) can reduce 
the high peak-to-average ratio (PAR) of multi carrier modulation signals. The clipping of the high 
peaks caused by analog circuitry leads to an impulsive additive noise. He showed that we can detect 
the positions of the noise impulses (by setting 95% of the clipping amplitude V c as a threshold), 
and then Analog Reed Solomon (RS) erasure decoding can correct the clipping errors. Another 
source of impulsive noise on multi carrier modulation signals are nulls in the channel's frequency 
response. It is well known that uncoded orthogonal frequency-division multiplexing (OFDM) must 
cope with symbol recovery problems when the channel has nulls close to or on the FFT grid. Wang 
and Giannakis [8] introduced complex field precoding for OFDM (CFC-OFDM) where a complex- 
field coding is performed before the symbols are multiplexed to improve the average performance 
of OFDM over random frequency-selective channels. They provided design rules for achieving the 
maximum diversity order, and showed that if the channel is modeled with random taps, a good 
choice of the (analog) precoding matrix can enhance the average BER and suits any realization of 
the channel coefficients. In |9] Henkel and Hu showed that OFDM can be seen as an analog RS code 
if a cyclically consecutive range of carriers is not used for transmission, and in ifTOll Abdelkefi et al. 
used the pilot tones of the OFDM system as a syndrome to correct impulsive noise in the presence of 
additive white Gaussian noise (AWGN). A different type of analog codes is presented in ifTTll . lfl2l . 
A linear space time block code is used to generate transmit redundancy over the real/complex field. 
However, these papers design optimal transmit redundancy for optimal Linear receivers and solve 
the coding problem under a MSE performance metric. Therefore, these coding designs are better for 
AWGN but not for impulsive noise. 

Another related topic is Compressed Sensing (CS). In Compressed Sensing we are given a repre- 
sentation dictionary D (defined as a compressed sensing matrix of size r x n) and the rows of D are 
used to sample the information vector x 



Given the vector s, which lies in the low dimensional space R r , we want to extract the information 
vector x, which lies in the higher dimensional space W n . Under the assumption that the vector s is 
composed of as few columns of D as possible, we look for the sparsest vector x that explains s. 
In other words, we are looking for the solution of equation ((5]) with the replacement of H with D. 
Under a certain condition on H and the size of the support of e, the sparsest solution of © can be 
found by minimizing the t\ norm instead of the £q norm El. |[T3l .|[T41.[ 15 1. 



s = Dx 



(6) 



mm 

XGK" 




subject to Hx = s (Pi)- 



(V) 



The i\ norm is convex and (0 can be solved using Linear Programming (LP) 0. 
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Donoho and Elad in |[T5l . Ifl6l and ifTTl introduced the term Incoherent Dictionary (or mutual 
incoherence property) which simply means that for every pair of columns of a dictionary D = 
[di,d 2 , . . . ,d n ] 

l(di,dj)|<^ (8) 

where p, is the coherence coefficient. They showed that in the special case where the CS matrices H 
are constructed by concatenating two unitary matrices $ and ^ of size rxr, the equivalence between 
© and © holds for ||x||^ < ^j ' 5 , where M is defined as M := sup { | {ipi , cpj ) | ,1< i,j < r}. In 
lfl6l it was shown that < M < 1. Hence, if the two unitary matrices and ^ are chosen such 

that M = 1/y/r (i.e. the coherence coefficient p = 1) the equivalence holds as long as ||x||^ < an 1 / 2 
for some constant a (it was shown in lfT3l that a « 0.65). 

Candes and Tao [2] introduced the term Restricted Isometry Property (RIP), which measures how 
orthogonal the columns of H are. Let H £ M rxn , T be a subset of {1,2, ...,n} and Ht be a 
submatrix of H, constructed by taking the columns of the matrix H indexed by T. The restricted 
isometry property of order L is defined as the smallest number 5l such that for all \T\ < L , c G M' T ' 

(1 - S L ) \\c\\l < \\H T c\\l < (1 + S L ) \\c\\l . (9) 
It is easy to show (see O) that if A (A) is an eigenvalue of the matrix A then (O is equivalent to 

1 - 5l < A (H^Ht) < 1 + 5 L V \T\ < L. (10) 
The RIP is important since if the RIP constants satisfy 

6t + 5 2 t + 5 3t <l (11) 

then problems ((5]) and ^} are equivalent when the size of the support of e is at most t. Therefore, if 
H has a "good" RIP one can correct any t errors using Linear Programming. The Gaussian random 
matrices satisfy (fTTT ) for ||e|| £ < pn (p << 1) with a probability of 1 — e(n) where e(n) decays 
exponentially to zero with n. 

In some applications, deterministic matrices with the Restricted Isometry Property are desirable 
due to storage limitations. However, there is no deterministic construction of a matrix for which 

(fTTT) holds with a constant fraction of the block length. DeVore lfT8l used a polynomial construction 

i 

to obtain a matrix that satisfies RIP for ||e||^ < an™+! where m > 1 is an arbitrary integer. The 
matrices of ifTSl provide a higher code rate than the matrices in Donoho and Elad lfT5l - lfT6l . which 
have a rate of R = ~ = 0.5, but DeVore's matrices correct fewer than a^fn errors. Note that when 
the RIP fails, there is no guarantee that the l\ minimization |7]) will compute the sparsest solution. 
Unfortunately, verifying the RIP for a given matrix H is a difficult task with exponential complexity. 
This property requires checking (O for all sub matrices of selecting t arbitrary columns. Lee and 
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Bresler |[T9l used the l\ relaxation and some additional relaxations to verify the RIP in polynomial 
time by using Semidefinite Programming (SDP). Statistical versions of the RIP (STRIP for short) were 
introduced by Gurevich et al. |[20l and by Calderbank et al. EH. Both versions bound the probability 
that the RIP holds for an L-sparse random vector (i.e. the L entries of the vector chosen at random). 
EOl showed that the STRIP holds in general for any incoherent matrix. In [21] Calderbank et al. 
bound STRIP'S performance for a large class of deterministic complex matrices. More specifically, 
they showed that under the assumption that the matrix H £ C rxn has columns that form a group 
under point-wise multiplication and rows that are orthogonal and vanish under summation (the row 
sums are equal to zero), the RIP © holds for 1 > 5l > ^Ej for any L-sparse random vector x with 
a probability of 

2L i 2L+7 

P RIP (X) = 1--^ 2=3-,. (12) 

(fc - S3) 

It was pointed out in ll22l that this assumption is too weak since almost all linear codes meet 
these conditions (for example a partial DFT matrix when excluding the first row), however it is 
not guaranteed that they will perform well for compressed sensing or equivalently for decoding 
linear analog correcting codes by the l\ minimization (0. In [22] Gan et al. showed a tighter 
bound on the performance of the STRIP in the case of matrices that nearly meet the Welch bound 
(which is a stronger restriction on the dictionary that bounds the mutual coherence of the matrix 
H - max | (hi, hj) |). It has been shown that for these matrices the RIP holds with probability that 
exponentially decays with (r/L). 

In |[23l Turbo codes were first introduced. Their performances in terms of bit error rate (BER) are 
close to the Shannon limit. In |[24l a coding scheme of block turbo codes (BTC) was described, where 
two (or more) encoders are serially concatenated to perform a product code. The product codes are 
used in the area of digital error correction codes (i.e., codes over a finite field) and are very efficient 
for building long block codes by using several short blocks. The decoding of such codes can be 
done sequentially, using one decoder at a time. In E51 . E51 analog products codes are presented. The 
iV 2 -lengfh information sequence is reshaped into an N x N information matrix. Then, the encoding 
is done by adding a parity check component to each columns and row of the information matrix 
such that the columns and rows of the (N + 1) x (N + 1) encoded matrix are sum to zero. This 
process describes a product code with analog parity check component. The decoding was done using 
an iterative decoder that converges to the least squares solution. However, in contrast to the method 
described on this paper, these product codes are optimized for MSE criterion instead of l\ criterion 
required for sparse reconstruction. 

As with the statistical version of the RIP, in this paper we weaken the strong RIP constant 
requirement at the price of an arbitrary small probability of error. However, in contrast to STRIP, 
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in this effort the problem of decoding the long block code is decoupled into two sets of parallel 
Linear Programming problems, which leads to much lower complexity than solving ([7]) to decode the 
codeword at once (see Hill) . In other words, the reconstruction of e from y is performed using LP 
(iteratively) even though ||e||^ is higher than what is required by the RIP, with the caveat that for a 
few ensembles of errors the reconstruction fails. More specifically, inspired by the iterative decoding 
of Turbo block codes |24l we show that given a code capable of correcting a^/n errors, we can 
construct a turbo analog block code that is capable of correcting up to a ^ g ^ errors with a probability 
of 1 — e(n) where e(n) decays sub exponentially to zero. This provides a simple analog coding 
procedure that improves existing deterministic coding matrices by using a probabilistic approach. 

The outline of the paper is as follows. Section [TT] describes the analog product code and a mathe- 
matical formulation of the problem. Section [III] gives the solution and a bound on the probability of 
decoding and the complexity of this turbo analog decoder. Section [TV] provides simulation results for 
the extended Donoho matrices described in section [Till We end up with some conclusions. 

II. ANALOG PRODUCT CODES AND PROBLEM FORMULATION 

Suppose that we want to encode a vector m G R. k , where k = K 2 K G N. Suppose that we reshape 
the vector into a matrix M G R KxK . Assume we are given a code generator matrix G G M NxK . Let 
Ri = be the code rate of G. The analog product coding process is as follows: 

1) inner code - code each column of M using the coding matrix G to produce a new matrix 

M G R NxK . 

2) outer code - code each row of M using the coding matrix G to produce a new matrix Y G 

Let R = ^ = R 2 be the code rate of the analog product code, where n = N 2 . This process can be 
written more compactly as 

Y = MG T = GMG T , (13) 

where G T is G transpose. It easy to see from (fT3l) that the order of the two stages above is irrelevant. 
As in section [IJ we assume that the model is Y = Y + E. Therefore, Y = GMG T + E, where 
E G M^" Xv/ " is the arbitrary (sparse) error vector presented as a matrix. Since G has full rank, 
decoding M from Y is equivalent to reconstructing E from Y. By the linearity of the code, the 
parity check matrix H G M N_KxN such that HG = provides a set of equations that do not depend 
on the input matrix M. 

HY = HGMG T + HE = HE 
YH T = GMG T H T + EH T = EH T (14) 
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Denote \\A\\ e := j) : Aij / 0}|. The decoding problem becomes 

min IIEIL s.t H(Y|Y T ) =H(E|E T ). (15) 

III. The probability of error for the two step iterative LP Turbo decoder 

In this section we show that any code that is capable of correcting up to a^fn errors can be 
extended by the scheme of Turbo codes to a code correcting up to °"^ 4 with a probability of error 
going to zero as a function of n. Let G be a generator matrix of a code that is capable of correcting 
up to a^fn errors. Let the coding process be as shown in section HO The main theorem is that if 
||E||^ o < "ggn > one can And the solution to (U~5l ) with a probability approaching 1 as a function of n. 

To prove the above, we use a two-step decoding procedure. First we decode each row of Y 
independently using (|7]) and correct the errors found in this step, then decode each column of the 
corrected matrix in the same way. Then, we bound the probability of error of the two-step decoder 
and show that the bound decays to zero (sub exponentially) as the block size increases. 

For the decoding process we use the following notation, given a matrix A. We denote the j'th 
row of A by (A T )j and the i'th column of A by Ai. The decoding process of the outer code is as 
follows. Let E be the error of the outer code. Each row of E can be found by solving ([7]) for each 
row of Y sequentially: 

(E T )j = arg mh ||Xj||^ s.t H(Y T )i = Hx ; i = l,...,N. (16) 

Following the notation of (fT3T ). this gives us M, the decoded matrix of the outer code, M G M NxK . 

The decoding process of the inner code is done as follows. Let E £ M Ar><i ^ be the error of the inner 
code, M = GM + E. Each column of E can be found by decoding each column of M sequentially. 

Ej := arg min ||Xj|| £i s.t H(Y - E) ; = Hx ; i = l,...,K. (17) 

The main theorem is that this two-step decoder correctly decodes the codeword and gives the sparsest 
solution of (U~5l) with a probability approaching one sub exponentially with n. 

Moreover, the problem of decoding the long block code is decoupled into two sets of parallel 
Linear Programming problems. This decoupling leads to a lower complexity than solving ([7]) to 
decode the codeword at once. More specifically, decoding a long codeword with size n using Linear 
Programming as in {/J takes 0(n 3 5 ) operations (27j. For the outer decoder, each row is decoded 
using LP with 0(N 3 - 5 ) operations; there are iV such rows. For the inner decoder each column is 
decoded with 0(iV 3 5 ); there are K such columns. Using the relation n = N 2 and assuming N K 
the iterative decoder decodes with only 0(n 2 25 ) operations. 

Again from (fT3l) it is easy to see that the decoding procedure can be done in the reverse order; 
i.e. first decode column by column and then row by row. Because the constraints are independent the 
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decoding procedure can be rewritten as 

E:=aro min V b; ,-| s.t HY T = HX T (18) 

hi 

B:=arg min \bii\ s.t HM = HB (19) 

hi 

where M can be found by solving 

MG T = Y — E (20) 

To intuit why this two step decoder leads to the solution of (fT3T >. consider a scenario in which 
only the first row has more than aN 1 / 2 = an 1 ^ errors. Assume the worst case is that if a codeword 
is decoded erroneously every entry of the word is wrong. After we decode row by row as in (fT6l ). 
every row except the first one will be error free (since the code is capable of decoding up to aN 1 / 2 
errors). Thus N errors shift to the inner code such that there is only a single error in each column. 
This can be corrected by decoding the columns as in (fTTT ). 

One should bear in mind that if the number of errors is bounded by t(n), the worst case for the 
two-step decoder is that there is no row that is completely filled with errors. Suppose that the total 
number of errors on the block is t(n) and a certain row has ti > an 1 ' 4 " errors; without loss of 
generality assume it is the first row. Thus, the rest of the block has t(n) — t\ errors. After decoding 
row by row, we assume that the first row is decoded with errors no matter how large t\ is (because 
t\ > an 1 / 4 ). Therefore, for larger t\, there are fewer errors left for the rest of the block and it has a 
higher probability of being decoded without errors. 

Lemma 1: Given a code that is capable of correcting ay/~N errors, the decoding procedure described 
by (fT8T)- (l2"0"l) provides a complete burst protection for bursts with sizes up to ib(n) := an 3 / 4 — n 1 / 2 + 
2cm 1 / 4 + 1 for any block size n = N 2 (under the assumption that there are no other errors on the 
decoded block). 

Proof: Assume the vector y corrupted by t^n) consecutive errors. Since i&(n) = n 1 / 2 (an 1 / 4 — 
l)+2an 1 / 4 +l, reshaping the vector y into a matrix with size n 1 / 2 xn 1 / 2 causes there to be (an 1 / 4 — !) 
rows that are completely filled with errors, and two other rows that together have 2an x / 4 + 1 errors. 
After decoding the outer code as in (fT8l ) there will be no more than an 1 / 4 rows with errors. In other 
words, there will be no more than an 1 / 4 in each column. Therefore, the inner decoder (fT9l correct 
all the errors, and we decode the block correctly. ■ 
Theorem 2: Let G G M KxN be a generator matrix of a code that is capable of correcting a\/N 
errors, and let n = N 2 . The two-step decoding procedure described by ([T8T)-(l2"0l provides a turbo 
analog block code that is capable of correcting up to t{n) := ""^ 4 errors with a probability of 
1 — e(n), where e(n) decays sub exponentially to zero with n. 
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Proof: The code fails to recover the correct word if the number of codewords that are decoded 
with errors in the outer code is higher than an 1 / 4 . In other words, if there are more than cm 1 / 4 rows 
with more than an 1 / 4 errors, the code will fail to recover the correct word. We want to bound the 
probability of that event. By assumption there are t(n) errors and N = ^fn rows. Set a random i.i.d 
binary process xi with 

p:=P(ii = l)4 = T' i = 1,2,..., t(n). (21) 
TV \/n 



t(n) 

Let y = ^2 x i be a binomial random variable with probability p. This is expressed as, 

i=i 

y~B(t(n),p). (22) 

Therefore, the probability that a given row will have more than an 1 / 4 errors can be bounded by the 
Chernoff bound. 

P{y > an 1 / 4 ) < e ~ w/4 ( pe s + 1 - pf n) . (23) 



Taking the derivative of the RHS of (1231 and equating to zero leads to 

(1 — p)an 1 / 4 
p(t(n) - an 1 / 4 ) 



^ = log L , 777T • (24) 



Where s > if 

an 1/4 < t(n) < an 3/4 . (25) 



Choosing 



an 3 / 4 



*M = r^T' (26) 
log(n) 



it is shown in appendix [A] that for all n > 2 

p(y > an 1/4 ) < q(n) (27) 

where 



-an 



1/4 



:= I — I (%(n))^T . (28) 



log(n) 
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Further simplification yields: 

q[n) < e v v 2 ,o9( "' ^ . (29) 

Therefore, since the total number of errors is bounded by t(n), we can uniformly bound the probability 
that a given row will be decoded with errors by q(n). For the inner code, we want to bound the 
probability that more than an 1 / 4 rows are decoded with errors. Assume the worst case that if a row 
is decoded with an error then all elements in the row are wrong. Denote the number of rows with 
errors by Z. We uniformly bound the probability that a given row is decoded with errors by q(n) 
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. Define a random i.i.d binary process y; L with P{yi = 1) = q , i = 1,2, ...,n 1//2 , set Z = yj a 

i=l 

binomial random variable with a probability of p = q(n). Therefore, P(Z > an 1 / 4 ) < P(Z > an 1 / 4 ) 
which can be bounded by the Chernoff bound. Choosing 

(1 — q)an 1 ^ \ 



s = loe 



(30) 



? ( n l/2 _ an l/4) J ■ 

Since a < 1, there exists a number Nq (typically a small number) such that for all n > Nq we have 
s > 0. A simple computation yields (see appendix iBl): 



P(block error) < Pf, 



where 



Pb ■= e 



-a 2 r i 1 /2 (loglog(n) _| + I 



-1/4) n-V*log(a^±-l) 



(31) 



(32) 



This bound decays sub-exponentially in the block size n. Therefore, the two-step decoder described 
by (TT8T)-(l2"0l finds the sparsest solution of (TT5l) with a probability of error decaying to zero as in (l32l 
when using Linear Programing. ■ 



Outer code probability of error 




2000 4000 6000 8000 10000 12000 14000 16000 



Fig. 1. Upper bound on the outer code probability of error, equation j29\ 



IV. Numerical Experiments 

In this section we investigate the performance of the two-step decoder in two sets of simulations. In 
the proof of the main theorem we uniformly bound the probability that a given row is decoded with 
errors. Therefore, in the first set of simulations we check the tightness of this bound. The number 
of errors t{n) = af^; is fixed and we check how frequently a "bad" ensemble of errors has been 
chosen in vectors with a support size of t(n) selected at random, for various sizes of block n. By 
a "bad" ensemble we mean an ensemble of errors that has more than an 1 ' 4 rows with more than 
an 1//4 errors. The results are shown in table U for a = 0.65. 
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Probability of error 




2000 4000 6000 8000 10000 12000 14000 16000 



n 



Fig. 2. Upper bound on the probability of error, equation <3U 



TABLE I 

"BAD" ENSEMBLE FREQUENCY FOR t(n) = 



n 


lo&10 (-fbad ensemble) 


number of errors - t(n) 


81 


-1.6 


4 


441 


-4.21 


10 


1369 


-8.88 


20 


3481 


-11 


36 



In the second set of simulations we simulated the analog turbo block decoder that was shown in 
section [TTTl to recover M from Y = GMG T + E: 

1) N = 128. 

2) In the simulation we used Donoho matrix composed of an Identity matrix and a Hadamard 
matrix of size N/2 each. 

3) Take the support set of size t uniformly at random, and sample a vector e at size n = N 2 with 
i.i.d Gaussian entries on the selected support. 

4) Reshape e to a structure of matrix E of size N x N . 

5) Put Y=E (equivalent to choosing M = , there is no loss of generality since the code is 
linear). 

6) Reconstruct E from Y by solving equations (fT8l)-(f20b. 

7) Compare E to E. 

8) Repeat for various sizes of t ( 240 times for each t). 

The results are presented in figure [IV] Our experiment shows that the input is recovered all the time 
as long as ||e|| £o < 1500. Note that we prove that we correctly reconstruct E as long as ||e||^ < 97 
(put n and a = 0.65 as was shown in |[T3l for Donoho matrices). In other words, the simulation 
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results show that the actual performance of the Turbo analog scheme is much better than what has 
been proven. One explanation for this discrepancy is that Donoho's construction has been proven to 
guarantee correction as long as there are no more than an 1 / 2 errors, but some ensembles of errors 
can be corrected even though there are more errors than have been proven. A second explanation is 
that the uniform bound in the main theorem is very loose, as can been seen from table |U The third 
explanation is that in the proof of the main theorem we chose "" g 3 ^ 4 as the number of errors, but it can 
easily be shown that one can select and get a similar bound on the probability of error, where 
f(n) is a monotonically increasing function for all n > uq (for some large enough no). However, 
increasing the number of errors leads to a slower decay of the probability of error (see table [II] for 
the example of f(n) = log log n). 

Empirical frequency of exact reconstruction , n=16384 
1? ; X * * * *-*-* 
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Fig. 3. Reconstruction frequency of E with a support size of ||e|| £ from H(E\E T ) using iterative LP for decoding 



TABLE II 

"BAD" ENSEMBLE FREQUENCY FOR t(n) = ^ 



n 


lo&lQ (-^bad ensemble) 


number of errors - t(n) 


441 


-1.67 


34 


1369 


-2.13 


73 


3481 


-2.6 


140 


7225 


-3.5 


233 


13225 


-4.8 


356 



V. CONCLUSION 

In this paper we have presented a simple analog coding procedure that improves existing deter- 
ministic coding matrices by using a probabilistic approach. The proposed coding/decoding scheme is 
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able to correct up to "" g 3 ^ 4 errors by solving a set of LP problems iteratively. This scheme shows a 
significant reduction in decoding complexity as compared to one-step LP decoding. Here we weakened 
the RIP requirement by allowing a vanishingly small probability of error where a Chernoff bound 
on the probability of error shows a sub-exponential decay to zero with the increase in block size. 
Moreover, simulation results show much better performance by this scheme. 

Appendix A 

Chernoff bound on the probability of error of the outer code 

The probability that a given row will be erroneously decoded is bounded using the Chernoff bound. 
Let y be as in (l22l) i.e. y ~ B(t(n),p) where p = n^ 1 / 2 . Assume n > 2 

P(y> an 1 / 4 )) < e~ sanl/l E {e^} = Vs > 0. 
= e - sanl,i (pe s + 1 - pf n) 

Let 



Where s > if 



Choosing 



get 



-an 



1/4 



, _ 1/2 \ 

l ~ n 1 » Qog(n))"S5cST 



/(l-n-V»)]og(n)W»> an n , „-°" 3/4 
[ { l-n-Jlo g (n) ) 0°g(")) ^ = 

A(n) ( B (n)) anl/i - an ~ 1/ilog W C{n)- an ^ + ^ 



where, 



n 1 /2(i 09 ( n ))-i 

C(n) = (l-n" 1 / 2 )™ 172 < e 



(33) 



s = arg min e~ san (pe s + 1 - p) t(n) . (34) 

s>0 



r w/4 ( P e s + i-p) t(n) 

Taking the derivative of the RHS of (1331 and equating to zero leads to 

(1 — p)an 1 / 4 
p(t(n) - an 1 / 4 ) 

an 1 / 4 < t{n) < an 3 / 4 (36) 



J ■ (35) 



log(n) 

p(y > an 1 / 4 ) < q(n) 

1/4 , an 3 / 4 Q „3/4 a „3/4 

(l-n^ 1 / 2 ) (an 3 / 4 ) (iog(n)) <°9(«) 

(°- 3/4 ) Q " 1/4 (t^T-^ 1/4 ) _Q " 1/4+ft ^ (38) 



A(n) = (/o 5 (n))" anl/4 = e -«™ 1/4 MM™)) 

/ , \ -n 1 ' 2 (login))- 1 

B(n)= (39) 



DRAFT 



14 



B(n) monotonically decreases to e and for all n > 2 

B(n) < e 3/2 . (40) 

Therefore, 

p(y > an 1 ' 4 ) < q < e — 1/4 (l°gMn)-f +7^-^) (41) 



Appendix B 

Chernoff bound on the probability of error of the inner code 

In JII] we assumed the worst case that if a row decoded with errors, the entire row is wrong. 
Therefore, for bounding the probability of block error, we need to bound the probability that more 
than an 1 / 4 rows are decoded with errors. Denote the number of rows with errors by Z. We uniformly 
bound the probability that a given row will be decoded with errors by q(n) (see (|27T ) ). Define a 

random i.i.d binary process y^ with P(yi = 1) = q , i = 1,2, ..^n 1 / 2 , set Z = ^^yi the binomial 

i=l 

random variable with a probability of p = q(n). Therefore, P(Z > an 1 / 4 ) < P(Z > an 1 / 4 ) which 
can be bound by Chernoff 

P(Z > an 1 / 4 ) < e ~ sanl/i {qe s + 1 - q) n ^ (42) 

By the first derivative test of the RHS of (l42l) . one can find that 

, / (l-g)an 1 / 4 \ 
\ q(n l ' z — an 1 '^) I 

and it easy to show that since a < n 1 / 4 , there exists a number Nq (typically a small number) such 
that for all n > Nq we have s > 0. Therefore, 

P<"»ck error) < ^;^''ZZ'^7'" < 

< ( a n'/t" i;i (» 1/2 )°' / ' < 

— -nr, 1 / 4 / 1/2 IM^i, 1 / 2 -^ 1 / 4 — 

g an (n 1 /2_ Qn l/4) n c n 

an 1 ' 11 , _„i/2 ( 44 ) 



art 



1/4 /n 1 / 4 



II (1 - an V^j < 



q 

e -W (log log(n)-f+ '°^-°r 1/4) - "' 1/4l ° g ^' 1) ). 

Note that the exponent is negative for all n > No(a) (for example A^o = 3340 for a = 0.65); 
therefore the bound decays sub-exponentially in block size n. 
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